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Abstract 

Coronary computed tomography angiogra- 
phy (CCTA) is increasingly used for the 
assessment of coronary heart disease (CHD) 
in symptomatic patients. Software applica- 
tions have recently been developed to facili- 
tate efficient and accurate analysis of CCTA. 
This study aims to evaluate the clinical appli- 
cation of computer-aided diagnosis (CAD) 



software for the detection of significant coro- 
nary stenosis on CCTA in populations with low 
(8%), moderate (13%), and high (27%) CHD 
prevalence. A total of 341 consecutive patients 
underwent 64-slice CCTA at 3 clinical sites in 
the United States. CAD software performed 
automatic detection of significant coronary 
lesions (>50% stenosis). CAD results were 
then compared to the consensus manual inter- 
pretation of 2 imaging experts. Data analysis 
was conducted for each patient and segment. 
The CAD had 100% sensitivity per patient 
across all 3 clinical sites. Specificity in the low, 
moderate, and high CHD prevalence popula- 
tions was 64%, 41%, and 38%, respectively. 
The negative predictive value at the 3 clinical 
sites was 100%. The positive predictive value 
was 22%, 21%, and 38% for the low, moderate, 
and high CHD prevalence populations, respec- 
tively. This study demonstrates the utility of 
CAD software in 3 distinct clinical settings. In 
a low-prevalence population, such as seen in 
the emergency department, CAD can be used 
as a Compute r-Aided Simple Triage tool to 
assist in diagnostic delineation of acute chest 
pain. In a higher prevalence population, CAD 
software is useful as an adjunct for both the 
experienced and inexperienced reader. 



Introduction 

Coronary computed tomography angiogra- 
phy (CCTA) is increasingly used for the non- 
invasive assessment of coronary artery dis- 
ease in stable symptomatic patients. Several 
multicenter clinical trials comparing 64-slice 
CCTA with invasive catheter-based coronary 
angiography have shown remarkable diagnos- 
tic accuracy for the detection of obstructive 
coronary heart disease (CHD) (>50% steno- 
sis). 1,2 In addition, a negative CCTA has been 
demonstrated in multiple studies to improve 
diagnostic certainty for ruling out significant 
coronary artery disease in a low-risk acute 



chest pain population, as well as reduce hospi- 
tal length of stay and healthcare costs when 
compared with the standard of care. 3 4 

The increased utilization of CCTA has 
resulted in a need to rapidly interpret a large 
volume of images in an accurate and efficient 
manner. To meet this growing demand for 
expert image interpretation, computer-aided 
diagnosis (CAD) software applications have 
been developed to expedite the evaluation of 
coronary heart disease with CCTA. The multi- 
ple benefits of automated software programs 
have been previously demonstrated in many 
non-cardiac diagnostic imaging modalities. 5 7 
However, the clinical utility of CAD for the 
assessment of CHD is still in its infancy. While 
various software applications have been vali- 
dated for the automated evaluation of CCTA 
and automated aortic calcium scoring, a use- 
ful commercially available product has yet to 
be established. 8 9 

Here we evaluate a dedicated CAD software 
designed for the automated evaluation of 
CCTA: the COR Analyzer by Rcadia Medical 
Imaging. Recently, 3 studies were published 
which evaluated the COR Analyzer perform- 
ance and reported promising system sensitivi- 
ty and negative predictive values. 1012 
Furthermore, the 3 studies proved to have 
specificities of over 60%, which is significant 
when considering the potential for the soft- 
ware to be used as a Computer-Aided Simple 
Triage (CAST) tool. 13 A successful CAST tool 
can be employed in the emergency department 
as an initial rapid read to allow for triage of 
patients. This is a step forward from the typi- 
cal CAD role as a confirmatory second reader. 
However, the past studies evaluating the COR 
Analyzer were limited in their size as well as 
in their patient populations and further confir- 
mation of these results is necessary. 

The current study aims to further evaluate 
the applicability of CAD software for the detec- 
tion of significant coronary stenosis by 64- 
slice CCTA. We aimed to determine the utility 
and accuracy of the COR Analyzer compared to 
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manual CCTA analysis in various clinical sites 
using different types of equipment (CT scan- 
ners) and among populations with varying 
CHD prevalence. 



Materials and Methods 

Coronary CTA was performed on consecutive 
patients at 3 clinical sites within the United 
States: Emergency Room, NY (ER); Imaging 
Center, WA (IC); Outpatient Office, NY (OP). All 
CCTAs were acquired using one of three 64- 
slice multi-detector computed tomography scan- 
ners: 64-detector row Lightspeed VCT scanner 
(GE Healthcare, Milwaukee, Wisconsin); 64- 
detector row Sensation 64 scanner (Siemens 
Health Care, Forchheim, Germany); 64-detector 
row Aquilion-64 scanner (Toshiba Medical, 
Tochigi, Japan). The 3 sites were assigned a 
disease prevalence category based on the rela- 
tive prevalence to the other sites included in the 
study; ER was considered low risk (<10% preva- 
lence), IC was considered intermediate risk 
(11-20% prevalence) and OP was considered 
high risk (>20% prevalence). A standard CCTA 
protocol was employed at all sites. Patients were 
pre-treated with oral and/or intravenous beta- 
blockers as necessary to achieve a resting heart 
rate target of less than 70 beats/min. All patients 
received one 4 mg sublingual nitroglycerin 
tablet five minutes prior to the CCTA scan. 
Electrocardiographic (ECG) prospective trigger 
or retrospective gating was used for all scans. 
The scanning protocol was adjusted for patient 
weight and heart rate, but typically included: 
64x0.5 mm collimation, 80-120 kV tube voltage, 
400-700 mAs of current for prospective scans 
and current modulation during retrospective 
scans. The entire volume of the heart was 
acquired in 8-9 s during a single breath-hold. 
For axial images, an effective slice thickness of 
0.75 mm with a reconstruction increment of 0.5 
mm was used. Optimal images and reforma- 
tions were reconstructed on a high-resolution 
matrix of 512x512 pixels and during several car- 
diac phases (various percentages of the R-R 
interval) when available. 

Each CCTA study was independently inter- 
preted by 2 ACC/ACR credentialed level III 
CCTA imaging experts. Images were qualita- 
tively classified as excellent, good or subopti- 
mal studies. The CCTA studies were analyzed 
for the presence of significant stenosis 
(defined as >50%) using a combination of 
axial images, 3D volume-rendered images, 
multiplanar reformations, and maximum 
intensity projections. A 10-segment model of 
the coronary artery tree was used for per seg- 
ment analysis: left main and the proximal, mid, 
and distal segments of the 3 major coronary 
arteries, which corresponded to the CAD 
reporting scheme. Vessel segments less than 1 



mm in diameter were excluded. All CCTA stud- 
ies performed were included in the final analy- 
sis irrespective of study quality. 

Following manual interpretation, the CCTA 
studies were transferred to a separate worksta- 
tion equipped with the CAD software (COR 
Analyzer®, version 1.8.1070 by Rcadia Medical 
Imaging, Haifa, Israel). The CAD software per- 
formed automatic segmentation of the coro- 
nary artery tree, labeling of coronary arteries, 
and detection of significant coronary lesions 
(defined as the most severe lesion in each seg- 
ment with >50% stenosis). Figure 1 shows 
both normal and obstructive reads by the CAD 
software with the corresponding manual CCTA 
reads for reference. Since the discussion about 
image processing and machine learning algo- 
rithms used by the CAD system is beyond the 
scope of this journal, for detailed technical 
description we refer the interested reader else- 
where in the literature. 1418 

In addition to reporting a positive or nega- 
tive study the CAD system may also report a 
warning when there is an error in the auto- 
matic analysis or the system is not confident 
about the result. Warnings are issued for a 
specific blood vessel segment and are usually 
caused by various imaging artifacts, low image 
quality, or unrelated software errors in vessel 



tracking. A warning is not an indication for a 
lesion detected by the system, but rather an 
invitation for the user to pay extra attention to 
the specified area. In case of a warning on a 
negative study, the human observer should 
confirm the lack of findings and declare it neg- 
ative, while in the case of a warning on a pos- 
itive study, the reader should detect a lesion by 
inspecting the indicated area and declare it 
positive. Therefore, warnings on negative 
studies were not counted as false alarms and 
warnings on positive studies were considered 
as true positives (hits). 

The guidelines for the evaluation of diagnos- 
tic performance of CAD were provided by the 
interpretative recommendations in the CCTA 
consensus manual. The analysis was performed 
for each patient and segment; sensitivity, speci- 
ficity, negative predicative value and positive 
predicative value were calculated. Approval from 
the IRB committee at the coordinating site was 
obtained prior to study initiation. 



Results 

A total of 341 patients with appropriate indi- 
cations for 64-slice CCTA at 3 clinical sites in 




Figure 1. (Upper panels) MIP of a normal coronary computed tomography angiography 
(left) and a normal Rcadia computer-aided diagnosis image output (right) of the same 
patient. (Lower panels) MIP (left) and Rcadia computer-aided diagnosis image and find- 
ings output (right) of a coronary computed tomography angiography with 2-vessel non- 
calcified obstructive (>50% stenosis) disease. Red arrows point to the proximal LAD 
lesion and blue arrows point to the OM branch lesion. 
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the United States were included in the study. Of 
these, 96 patients underwent CCTA in the emer- 
gency room (ER) of a suburban tertiary medical 
center, 196 patients were scanned in a subur- 
ban outpatient cardiac imaging center (IC), and 
49 patients had CCTA at an urban outpatient 
cardiology office (OP). The prevalence of CHD 
(>50% stenosis in one or more vessels) in the 
population at these 3 locations was low (8%), 
moderate (13%), and high (27%) for ER, IC and 
OP, respectively. Two of the 341 CCTAs were 
characterized as suboptimal in quality, but were 
still included in the analysis. 

The CAD results categorized the CCTA as 
positive for obstructive CHD in 37 of the 96 
scans performed at ER, 122 of the 196 at IC, 
and 34 of the 49 in OP. In the segmental analy- 
sis, segments were classified as positive in 77 
of 960 ER segments, 278 of the 1960 IC seg- 
ments and 76 of 490 OP segments. The consen- 
sus manual interpretation found that 8 of the 
96 CCTAs at ER, 26 of the 196 CCTAs at IC, and 
13 of the 49 CCTAs at OP had findings consis- 
tent with obstructive CHD. On a per segment 
basis, 19 out of the 960 ER segments, 29 out of 
the 1960 IC segments, and 27 out of the 490 OP 
segments were found to be representative of 
significant coronary stenosis. In the per 
patient analysis, there were 7 warnings on 
negative studies in both the ER and IC sites 
and 0 warnings on negative in the OP site. In 
the per vessel analysis, there were 24 warning 
on negative studies in the ER site, 20 in the IC 
site, and 6 in the OP site (Table 1). 

The CAD had a per patient sensitivity of 
100% across all 3 clinical sites. Specificity in 
the low, moderate, and high CHD prevalence 
populations was 64%, 41%, and 38%, respec- 
tively. The negative predictive value (NPV) at 
the 3 clinical sites was 100% (this is by default 
when a test results in a sensitivity of 100% and 
a specificity >0%). The positive predictive 
value (PPV) was 22%, 21%, and 38% for the 
low, moderate, and high CHD prevalence popu- 
lations, respectively (Table 2). The segmental 
analysis yielded results with a similar trend 
(Table 3); however, the absolute values were 
lower due to misclassification in predominant- 
ly distal segments and segments affected by 
significant motion artifact. 



times for various diagnostic procedures only to 
have a normal result. Computer-aided detec- 
tion of CHD in the emergency setting has the 
potential to efficiently rule out CHD in acute 
non-MI chest pain patients, and thus can expe- 
dite the discharge of a large portion of patients 
who are in fact negative for CHD. While stud- 
ies have validated the potential for CAD soft- 
ware for evaluation of CHD, 89 to date, few 
studies have explored commercially available 
CAD software. 

A past study evaluating the diagnostic accu- 
racy of the COR Analyzer software by compari- 
son with quantitative angiography (QCA) 
reported a high sensitivity and negative pre- 
dictive value. 10 However, this study was limited 
by its small population (n=59) which was 
restricted to patients referred for QCA; thus, 



low- to intermediate-risk patients were not 
assessed. Additionally, a recent mid-sized, 2- 
center study and a large single center study 
compared the accuracy of the COR Analyzer 
software on 64-slice and 256-slice CCTA by 
comparison with manual interpretation of 
CCTA, again demonstrated a high sensitivity 
and negative predictive value. 1112 These 3 
studies compliment the current, so far largest, 
multicenter comparison of the CCTA CAD on 
64-slice CCTA compared to manual CCTA inter- 
pretation. 

The present study illustrates that the CAD 
system can identify abnormal coronary seg- 
ments when employed in 3 distinct clinical 
sites with varying incidences of CHD. There 
was good correlation between manual inter- 
pretation and CAD results for the exclusion of 



Table 1. Warnings on negative for each site, in both the per patient and the per vessel 
analysis. 



Coronary artery disease prevalence (%) 
Low (8%) Moderate (13%) High (27%) 

Emergency room Imaging center Outpatient office 



Per patient 
Per vessel 



7/96 
24/960 



7/196 
20/1960 



0/49 



Table 2. Sensitivity, specificity, positive predictive value (PPV), and negative predictive 
value (NPV) for the identification of significant stenosis (>50%) on coronary computed 
tomography angiography by computer-aided analysis when compared with manual visu- 
al interpretation on a per patient analysis in 3 populations of varying disease prevalence. 





Coronary artery disease prevalence (%) 1 




Low (8%) 


Moderate (13%) 


High (27%) 




Emergency room 


Imaging center 


Outpatient office 


Sensitivity 


100% (8/8) 


100% (26/26) 


100% (13/13) 


Specificity 


64.2% (52/81) 


41.1% (67/163) 


38.2% (13/34) 


PPV 


21.6% (8/37) 


21.3% (26/122) 


38.2% (13/34) 


NPV 


100% (52/52) 


100% (67/67) 


100% (13/13) 



PPV, positive predictive value; NPV, negative predictive value. 



Discussion 

With the introduction of CCTA computer- 
aided detection software for the automatic 
detection of significant coronary heart dis- 
ease, the potential for rapid discharge of 
patients who present to the emergency depart- 
ment with acute non-myocardial infarction 
(MI) chest pain is promising. Due to the low- 
risk nature of the acute non-MI chest pain 
patient, 18 patients often endure long waiting 
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Table 3. Sensitivity, specificity, positive predictive value (PPV), and negative predictive 
value (NPV) for the identification of significant stenosis (>50%) on coronary computed 
tomography angiography by computer-aided analysis when compared with manual visu- 
al interpretation on segmental analysis in 3 populations of varying disease prevalence. 





Coronary artery disease prevalence (%) 
Low (8%) Moderate (13%) High (27%) 
Emergency room Imaging center Outpatient office 


Sensitivity 


73.7% (14/19) 


82.8% (24/29) 


100% (27/27) 


Specificity 


93.1% (847/910) 


86.7% (1657/1911) 


89.0% (397/446) 


PPV 


18.2% (14/77) 


8.6% (24/278) 


35.5% (27/76) 


NPV 


99.4% (847/852) 


99.7% (1657/1662) 


100% (397/397) 



PPV, positive predictive value; NPV, negative predictive value. 



[Heart International 2013; 8:e2] 



9 



OPEN JT1 ACCESS 



Clinical investigation 



significant CHD across a spectrum of disease 
prevalence. Specifically there were no false 
negatives in any of the populations studied. 

While the high sensitivity and negative 
predicative value for CAD using CCTA is prom- 
ising, both the specificity and positive predic- 
tive value were rather low by human analysis 
standards. On the other hand, when comparing 
to other known computer-aided analysis sys- 
tems (e.g. for mammography, colonoscopy, 
lung nodules, chest X rays, etc.), the reported 
specificity and PPV (especially in the low-risk 
population, 8% prevalence) is extremely high. 
All other computer-aided diagnosis systems 
available today generate several false alarms 
on average per every case, yielding a close to 
zero per patient specificity and PPV. 

The system reported in this study, yielding a 
64% specificity for a low-risk population, falls 
into a different category of computer-aided 
analysis tools: the Compute r-Aided Simple 
Triage (CAST) recently introduced by 
Goldenberg and Peled. 13 As a CAST tool, it can 
be used for chest pain patient triage in the 
emergency department or for reading sequence 
prioritization in high-volume practices with 
low-risk populations. For moderate- and high- 
risk populations, the CAD system yielded lower 
specificities and cannot, therefore, be used as 
CAST in these groups. However, a CAST tool is 
most useful in the setting of the emergency 
department where rapid triage of patients is 
critical, and it is well documented that emer- 
gency departments are a low-risk population 
with regards to coronary artery disease. 19 
Therefore, CAD has the potential to significant- 
ly aid the experienced practitioner with accu- 
rate and efficient CCTA interpretations and 
maintain throughput, especially in high-volume 
emergency departments where a rapid evalua- 
tion for CHD is crucial. A recent study using the 
COR Analyzer software supports the current 
study's results in the low-risk population, 
demonstrating a 63% specificity, further sup- 
porting the usage of the COR Analyzer software 
as a potential CAST tool. 12 

Although the evaluated CAD system cannot 
be used as a CAST tool for moderate- (13% 
prevalence) and high-risk (27% prevalence) 
populations due to the subpar specificities and 
PPV, it still has a role in these categories of 
patients. In populations in which false posi- 
tives are common, inexperienced radiologists 
can still benefit from having the second reader. 
Furthermore, it is important to note that the 
experienced reader can also benefit from the 
CAD software in populations where false posi- 
tives reads by the CAD software are frequent. 
Every study reported as positive by the comput- 
er system is to be immediately reviewed by a 
human reader. Therefore, the most important 
characteristic for all practical purposes is not 
the stand-alone specificity/PPV of the comput- 
er, but the combined computer-human diag- 



nostic performance. In this regard, it is impor- 
tant to mention that the majority of false 
alarms produced by the system were related to 
calcifications, imaging artifacts, and vessel 
tracking errors. Those can be easily and rapid- 
ly discarded by a human reader, yielding high 
combined computer-human specificity and 
PPV. These results are supported by findings in 
the previous comparison of manual CCTA and 
automated CCTA. 11 - 12 

We should note that the moderate preva- 
lence population had results that are a cause of 
concern and that should be further discussed. 
The low-prevalence population has a very low 
PPV that is counterbalanced by the strong 
specificity in this population. Also, in the high- 
prevalence population, the low specificity is 
counterbalanced by an improved PPV. The 
moderate prevalence population saw compara- 
tive reductions in both specificity and PPV, 
bringing into question the utility of CAD for 
CHD in this population. However, considering 
the previous discussion regarding the nature 
of the false positive calls, there is still a possi- 
bility that CAD might be useful in the moder- 
ate prevalence population and, as such, more 
research must be conducted before eliminat- 
ing the role of CAD in this population. 

Furthermore, with the introduction of the 
320-slice CT scanner, we hope for a large 
improvement in the specificity and positive pre- 
dictive value of CAD for coronary heart disease, 
mainly due to its wider volume coverage result- 
ing in significantly less stacking artifact. 
Improvements in the software, including proper 
analysis of heavily calcified vessels, as well as 
improved vessel tracking, can further advance 
the diagnostic ability of the CAD system. 

Limitations 

The retrospective application of the CAD 
software is a limitation of this study. A 
prospective evaluation could assess objectively 
whether the combined use of CAD and manual 
interpretation is superior to manual interpre- 
tation alone, as has been previously demon- 
strated in other non-cardiovascular imaging 
modalities using CAD, such as mammogra- 
phy. 20 Additionally, a prospective study can 
explore the logistics and utility of applying the 
CAD as a CAST tool in a high-volume emer- 
gency department. An important aspect to con- 
sider when designing a prospective study is 
that the mean time for CAD (approx. 7 min per 
phase) exceeds the time required for manual 
interpretation if more than 2-3 phases are sent 
for the automatic analysis (mean reading time 
of an expert SCCT level III reader is approx. 5- 
10 min; Poon M, unpublished data). While this 
has the potential to be the rate-limiting step 
when employed in a combined prospective 
work flow, the ability to facilitate the triage 
process in a high-volume emergency depart- 



ment setting where the majority of the acute 
chest pain cases are normal is still promising. 
Especially when considering nights and week- 
ends when it may be difficult to reach an 
expert SCCT reader and long patient waiting 
times is the norm.Another important limita- 
tion is the utilization of an expert manual 
interpretation of CCTA as the reference stan- 
dard rather than the typical QCA. However, in 
the emergency setting, most clinical decisions 
are made upon manual CCTA analysis, as 
opposed to QCA, and thus a comparison of CAD 
using CCTA and manual CCTA is appropriate 
in this study. Furthermore, a previous study 
has confirmed that CAD utilizing 64-slice 
CCTA has high sensitivity and negative pred- 
icative value when compared to QCA that nice- 
ly complements the results of the current 
study. 10 

A final limitation to consider is the exclusion 
of large coronary side branches during the auto- 
mated analysis. At the launch of this trial, the 
software did not have the capability to analyze 
side branches and so significant lesions may 
have been missed. However, since the launch of 
this trial, the COR analyzer has been up-dated 
and now incorporates side branches in the coro- 
nary analysis with promising initial results. 11 



Conclusions 

In this multi-center, retrospective study, we 
have shown the utility of the CAD for analysis 
of coronary heart disease using CCTA. By 
applying the software in 3 distinct clinical 
sites, we demonstrated that the CAD system is 
compatible with scans from 3 major CT scan- 
ner manufacturers as well as in varying patient 
populations with different incidences of CHD. 

In a low-risk population, the CAD system 
shows as a CAST tool. This is particularly excit- 
ing when considering the low prevalence of 
CHD typically encountered in the emergency 
department. While both the moderate and high 
prevalence populations are not viable candi- 
dates for the usage of CAD as a CAST tool, the 
importance in these populations is not lost. 
First, the inexperienced reader can benefit 
from a CAD second reader in ruling out false 
positives the inexperienced reader might 
encounter. Additionally, an experienced reader 
can rapidly disregard many of the false posi- 
tives read by the CAD software leading to an 
efficient human-computer interface in the 
diagnosis of CHD using CCTA. 

Overall, computer-aided diagnosis in the 
evaluation of coronary heart disease is starting 
to become a reality. With this study, we have 
shown the commercially available CAD system 
to be a potential candidate to take on this role 
in various clinical sites with various inci- 
dences of coronary heart disease. 
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